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ABSTRACT 


The Marine Corps has recently embarked on the development of a Vertical 
Takeoff and Landing Unmanned Aerial Vehicle (VTUAV) to replace the aging Pioneer 
system. This thesis examines the critical elements this platform must possess in order to 
effectively support small units operating in urban environments. We address this issue by 
creating and exploring an agent-based simulation of a platoon conducting an urban patrol 
in a setting similar to those currently being encountered in Iraq. The platoon utilizes the 
VTUAV as an intelligence-gathering asset. 

We use an efficient designed experiment to generate data from the simulation 
scenario, and then use multiple regression and regression trees to relate the UAV 
capabilities to the patrol’s operational effectiveness. Our results suggest that the 
effectiveness of a VTUAV is greatly influenced by noise in the urban warfare 
environment. We use a loss function, along with the regression models, to identify UAV 
configurations that improve operational effectiveness yet are robust to uncertainties about 
civilian and insurgent behavior. The VTUAV must have high communication capability, 
as well as accurate sensing, in order to perform well across a range of environmental 


conditions. 
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THESIS DISCLAIMER 


The reader is cautioned that computer programs developed in this research may 
not have been exercised for all cases of interest. While every effort has been made within 
the time available to ensure that the programs are free of computational and logic errors, 
they cannot be considered validated. Any application of these programs without 


additional verification is at the risk of the user. 
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EXECUTIVE SUMMARY 


Although major combat in Iraq has ended, the task of securing the nation is far 
from complete. The military forces in Iraq have the difficult task of conducting stability 
and support operations (SASO) to root out any insurgent and terrorist activities in order 
for democracy to take hold. In order to win the hearts and minds of the people, and 
stabilize the fledgling democracy in Iraq, the U.S. troops must focus on the political and 
population centers of Iraq. Unfortunately, it is in this urban environment that military 
operations can exact a heavy toll. Urban areas allow insurgent groups to inflict high 
casualties while avoiding decisive engagements. Urban areas also offer a target-rich 
environment for terrorist tactics, such as the kidnapping of civilian contractors, 
diplomats, and aid workers, as well as the use of improvised explosive devices (IEDs). 
Due to the success of these tactics, the U.S. must find better ways to utilize its 


technological and firepower advantages. 


During the course of the last year, the Marine Corps has introduced many 
technological enhancements in order to give our troops the best possible advantages. In 
addition to ground equipment improvements, the Marine Corps is currently seeking to 
replace the aging Pioneer Unmanned Aerial Vehicle (UAV) with a more capable platform 
better suited for an expeditionary force. This new UAV will be equipped with Vertical 
Takeoff and Landing (VTOL) capability. With this new capability the Marine Corps will 
have greater flexibility in the areas that the UAV could be employed. The new VTUAV 
will able to span the spectrum from low to high intensity combat, including urban 
warfare. Because of the dense chaos of urban warfare, we must examine the capabilities 
that a UAV must possess in order for U.S. forces to exploit this technological advantage 


in such an environment. 


In this thesis, we examine the impact of providing a small infantry unit with a 
UAV during urban patrolling operations. We create a simulation model of this scenario 
using an agent-based modeling platform called Map Aware Non-uniform Automata 
(MANA). This scenario is modeled off of the current situation our forces are 


encountering in Iraqi urban areas. The platoon has one UAV that is used as an 


XIX 


intelligence gathering asset. Enemy forces operate in small groups and utilize the cover 
and concealment of the urban landscape. The insurgent forces also exercise influence 


over the local populace, fueling anti-American sentiment. 


We gain insights from this simulation by using the technique known as Data 
Farming. Data Farming involves generating a very large number of data points from 
simple computer models, based on a wide range of model inputs, in order to explore the 
models’ behavior. These inputs include UAV capabilities: its sweep width, observation 
cover, and speed, along with communication reliability, latency, and accuracy. 
Characteristics of the Iraqi civilians and insurgents are also varied over broad ranges. We 
use an efficient experimental design approach to specify the combination of model 


inputs. Even so, over 40,000 simulations are run to provide data for our analysis. 


We use a robust design approach to analyze the results. This approach is based on 
the idea that a system should be relatively insensitive to variations or noise in the 
environment. Robust solutions are particularly appealing for combat systems. In the 
combat environment, military systems are constantly affected by uncontrollable sources 
of variation. This thesis is intended to be a preliminary study of the factors that 
contribute to the success of the VTUAV in support of a small unit conducting urban 
operations. Exact numerical results of this study should not be taken as a literal 
translation into real world numbers. It is the insights, not the numbers, that are the focus 


of our investigation. 


Based on the robust design philosophy our thesis work suggests the following 
results: 


° The hostility of the civilian population plays an important role in urban 
combat. If civilian hostilities are high, a more sophisticated UAV might 
be necessary. In environments with friendlier populations, other less 
sophisticated intelligence assets might be sufficient to accomplish the 
mission. 


e A platform that has very reliable communication capability and very 
accurate sensing capability is best suited for the range of environmental 
conditions modeled. These two factors were the most dominant in 
providing favorable results. 


XX 


e The aggressiveness of the blue force has an impact on the mean exchange 
ratio. In situations where commanders cannot tolerate the possibility of 
high losses, a less aggressive force that relies more heavily on the UAV 
for situational awareness and enemy engagements might be more 
appropriate. 


° The ‘best’ decision resulting from the robust design approach is different 
than the decision that would have resulted by only using mean 
performance. The resulting system is more insensitive to uncontrollable 
sources of variation in the combat environment. 


e By using intelligent design techniques, we are able to capture the essence 
of the problem by using relatively few data points. 


e The data farming environment, coupled with agent-based models, provides 
an excellent framework for gaining insights into interesting interactions in 
combat modeling. 


A UAV that has reliable communication and accurate sensing capability will 
work best across a broad range of environments. If we have some information about the 
environment such as the civilian hostility, we may be able to use a less sophisticated 
UAV and still operate effectively. Finally, the aggressiveness of the blue forces plays a 
key role in achieving higher exchange ratios, but it can also result in higher risks. If the 
potential for high losses can not be tolerated, the blue force should rely more heavily on 


the intelligence gathering and targeting capabilities of the UAV. 
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I. INTRODUCTION 


Intelligence is imperative to success in urban warfare. “Maneuver 
warfare requires a firm focus on the enemy” (MCDP 2, Intelligence). Few 
subsequent tactical changes can overcome the far-reaching impact of a 
major intelligence error. 


MCWP 3-35.3 
Military Operations in Urban Terrain (MOUT) 


A. MOTIVATION 

In the three years since the attack on the U.S. on September 11, 2001, the U.S. 
military has removed the ruling government from two nations and has begun the process 
of establishing democratic governments in both. In Iraq the military forces have the 
difficult task of conducting stability and support operations (SASO) to root out any 
insurgent and terrorist activities in order for democracy to take hold. They also must win 
the support of the people through various humanitarian projects and daily interaction. In 
order to win the hearts and minds of the people and stabilize the fledgling democracy in 
Iraq, the U.S. troops must focus on the political and population centers of Iraq. 
Unfortunately, it is in this urban environment that military operations can exact a heavy 
toll. Utilizing the advantages of the urban terrain, insurgents have begun a campaign of 
terror aimed at the U.S., its coalition partners, and even Iraqi citizens. With its superior 
combat power and technological advantage, the U.S. forces toppled Saddam Hussein’s 
regime in just three weeks. However, stabilization efforts are still ongoing over one year 


later. 


Urban areas allow insurgent groups to inflict high casualties while avoiding 
decisive engagements. Urban areas also offer a target-rich environment for terrorist 
tactics, such as the kidnapping of civilian contractors, diplomats, and aid workers, as well 
as the use of improvised explosive devices (IEDs). Due to the success of these tactics, 
the U.S. must find better ways to utilize its technological and firepower advantages. 
During the course of the last year the Marine Corps has introduced many technological 
enhancements in order to give our troops the best possible advantages. In addition to 
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ground equipment improvements, the Marine Corps is currently seeking to replace the 
aging Pioneer Unmanned Aerial Vehicle (UAV) with a more capable platform better 
suited for an expeditionary force. Because of the dense chaos of urban warfare, we must 
examine the capabilities that a UAV must possess in order for U.S. forces to exploit this 


technological advantage in such an environment. 


B. PURPOSE 

The purpose of this thesis is to utilize an agent-based model (ABM) to develop a 
simulation representing a small unit urban infantry patrolling operation utilizing a UAV. 
The simulation will be used to identify important factors, primarily related to the UAV, 


that are vital to mission success. 


C; BACKGROUND 

As part of naval expeditionary forces (NEFs), the Marine air-ground task forces 
(MAGTFs) are forward deployed for rapid crisis response. As a rapid response force, 
Marines may be employed in a wide range of military operations. Studies have shown 
that 75% of politically significant urban areas, other than allied or former Warsaw Pact 
territories, are located within 150 miles of a coastline. As a rapidly deployable naval 


force the Marines must be prepared to operate in these large urban areas. 


Historically, the attacker usually wins in urban battles. This statistic may be 
attributable to the fact that urban warfare extracts a heavy toll on the attacker, causing 
him to attack only when he has a decisive advantage—not only in numbers, but also in 
other factors such as intelligence and superior training and equipment. 

Regardless of the size or quality of defensive forces, the defender usually extracts 

large costs from the attacker in time, resources, and casualties. [MCWP 3-35.3, 

April 1998] 

Military operations in urban environments have historically involved small infantry units 


and have been casualty intensive affairs. 


Current operations in Iraq demonstrate the difficulties of patrolling in an urban 
environment. Insurgents are able to blend into the local populace, containing many who 
are sympathetic to their cause. Using IEDs and taking hostages are some of the tactics 


being used against U.S. Forces. In Ramadi, a city of about 400,000 on the Euphrates 
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River west of Baghdad, Marines are currently conducting security patrols looking for 
weapons and insurgents while still trying to win the hearts and minds of the local 
populace. During routine patrols in early April 2004, the city erupted in a series of 
coordinated ambushes by hundreds of Iraqis shooting at Marines from alleys, behind 


windows and doors, and out in the street. 


When this was taking place, two hours into it, everyone and their mother 
was shooting at us. 


Staff Sgt. Damien Rodriguez 
Regarding the fighting in Ramadi, Iraq April 6, 2004 


It is this type of fighting that has served as the basis for development of the scenario 


examined in this thesis. 


D. UNMANNED AERIAL VEHICLES BACKGROUND 

A Marine Corps analysis on the Russian war in Chechnya stated that UAVs were 
used extensively and effectively in the urban environment [FM 3-06.11, 2002]. The use 
of UAVs in military operations has stretched for nearly one hundred years. From the 
early years of flight, through the Vietnam War, to today, unmanned surveillance 
continues to play an important role in military operations. Because there are many 
different types of unmanned aircraft, we use the definition of a UAV from Joint 
Publication 1-02: 

A powered, aerial vehicle that does not carry a human operator, uses 

aerodynamic forces to provide vehicle lift, can fly autonomously or be 

piloted remotely, can be expendable or recoverable, and can carry a 

lethal or non-lethal payload. Ballistic or semi ballistic vehicles, cruise 


missiles, and artillery projectiles are not considered unmanned aerial 
vehicles. 


With today’s technology, the UAV is far more capable and sophisticated than those 
of just a few years ago. As technological advances continue, the UAVs of tomorrow will 
possess capabilities that will allow military commanders to receive large amounts of 
relevant information more quickly and more reliably than is possible using the UAVs of 


today. 


America’s first version of a modern UAV was the Lightning Bug. It was used as a 
targeting decoy during the Vietnam War. During the war it was outfitted with primitive 
computers and cameras for surveillance, but limitations in its navigation system meant it 
could not be reliably deployed. It continues to hold the record as the most expensive 
drone ever built by the U.S. In today’s dollars it would cost $4.5 billion [Spies That Fly, 
July 2004]. 


The next major advancement in UAVs occurred in 1982 during the Arab/Israeli 
War in southern Lebanon. Initially, the Israeli Air Force sustained significant losses due 
to the Soviet supplied anti-air capability. In response to these losses, the Israelis 
developed a UAV to act as a decoy. This propeller-driven UAV had no firepower, and 
was controlled from the ground with radio signals. It was slow, unsophisticated, and had 
limited range, but it carried signal generators that gave off the radar signature of a much 
bigger plane. By sending in a wave of these UAVs before the manned fighters, the 
Israelis caused the Syrians to turn on their radar. This allowed the Israeli fighters to 
launch anti-radiation missiles and destroy the radars. Later, the Israelis would equip 


these UAVs with cameras for surveillance operations [Spies That Fly, July 2004]. 


Motivated by the success of the Israelis and such acts as the bombing of the Marine 
barracks in Lebanon, the U.S. began to develop its own UAV (called the Pioneer) 
modeled after that of Israel. The Pioneer first saw action during Operation Desert Storm. 
Capable of staying aloft for hours, it was assigned to battle ships for targeting purposes. 
It was launched from the ship with a rocket-propelled catapult and retrieved in a cargo 
net. The Pioneer enabled the U.S. to receive real-time target damage assessment and 
reduced the need for human eyes on the target. Iraqi soldiers actually tried to surrender 
to these UAVs because they knew that if they heard a drone that bombs would usually 
follow [Spies That Fly, July 2004]. The Pioneer had a limited range due to its line of 
sight communications requirement. This challenge was soon answered with the 


introduction of the next U.S. UAV, called the Predator. 


The Predator was the first UAV that could receive signals from satellites and 
therefore know its position at all times. It could stay aloft for 40 hours. It first saw 


service in 1995 over Bosnia [Spies That Fly, July 2004]. It was equipped with a camera, 
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infrared radar (IR), Synthetic Aperture Radar (SAR), and a laser designator for targeting. 
The Predator is good at keeping track of a target, but its narrow field of view limits its 


ability to find a target unless the location is known. 


The largest and most sophisticated U.S. UAV is the Global Hawk. It was used in 
Afghanistan while still in the flight test stage. It is jet powered and can cruise at an 
altitude of 65,000 feet [Spies That Fly, July 2004]. The Global Hawk is programmed 
from takeoff to touchdown and does not have the capability of being piloted from the 
ground. It is a pure surveillance platform, carrying no weapons payload. It recently 
completed a flight from California to Australia without refueling [Spies That Fly, July 
2004]. The Global Hawk is capable of surveying a much larger area than the Pioneer, but 


at $40 million apiece it costs substantially more than any other current UAV. 


The Marine Corps currently possesses two UAVs that are both used for 
surveillance purposes only: the Pioneer and the Dragon Eye. The Dragon Eye is a man- 
packed, propeller-driven UAV. It is equipped with a small camera, has a short range and 
flying time, and is used for over-the-hill reconnaissance only. The Marine Corps has 
embarked on a program to replace the Pioneer with a Vertical Takeoff and Landing UAV 
(VTUAV). The VTUAV is expected to provide support to operations spanning the 
spectrum of Military Operations Other Than War (MOOTW) to a large intensity conflict 
[Operational Requirements Document (ORD) for the VTUAV]. Some of the limitations 
of Pioneer include its limited shipboard compatibility, obsolete Ground Control Stations 
(GCS) and inadequate speed. The VWTUAV will provide the capabilities to support the 
Marine Corps concepts of Operational Maneuver From the Sea (OMFTS) and Ship to 
Objective Maneuver (STOM). The capabilities of the VTUAV will include: target 
acquisition, surveillance, reconnaissance, target designation, communications and data 
relay, electronic warfare, and delivery of remote sensors and non-lethal weapons [ORD 
for the VTUAV]. The VTUAV will be an organic asset of the Marine Air Ground Task 
Force (MAGTF) Commander. Current fielding projections have the VTUAV available to 
the FMF by FY08 [ORD for the VTUAV]. 


E. SCOPE 

In an urban combat scenario it is possible to examine hundreds of factors that 
might contribute to mission success. The introduction of an intelligence-gathering asset 
such as a UAV to this environment adds to the complexity of the analysis. In order to 
conduct a practical examination of issues pertinent to the development and use of the 
VTUAV in conjunction with small unit urban operations, this thesis will adhere to the 


following plan for analysis: 


e Review urban warfare tactics. 

® Understand potential enemy tactics and courses of action. 

° Identify measures of effectiveness. 

° Explain the capabilities and limitations of the modeling tool. 

e Explain the development of the simulation scenario. 

e Discuss data farming and its application to the urban patrolling model. 

° Analyze the data from our simulation runs and fit models to the output. 

° Examine the results of our analysis as it pertains to urban patrolling with 
the VTUAV. 


F. AGENT-BASED MODELS 

This thesis will gain insights to questions by using the technique known as Data 
Farming. Data Farming is a method that generates a very large number of data points 
from simple computer models, based on a wide range of model inputs. The analyzed 
results can yield surprises, a better understanding of the model performance, and 
interesting outcomes. The Marine Corps Warfighting Laboratory’s Project Albert has 


provided the necessary infrastructure for this research capability to the Marine Corps. 


Project Albert’s Data Farming approach is achieved through the use of agent- 
based models or distillations. Agent-based models have entities that are controlled by 
decision-making algorithms, enabling the analyst to examine how agents interact with 
one another in potentially interesting and non-linear ways. These are simple models that, 
when coupled with high performance computing, can generate potentially millions of 
data points for analysis. The agent-based modeling platform used in this thesis is Map 
Aware Non-uniform Automata (MANA). MANA was developed by the New Zealand 
Defence Technology Agency and was inspired by the work of Andy [anchinski and 
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others using agent based models [Galligan et al.,2004]. MANA examines the interactions 
among agents that have their own personality traits and are able to move autonomously. 
The parameters for the agents fall into four basic types: (1) personality, (2) movement, 


(3) capabilities, and (4) movement constraints [Galligan et al., 2004]. 


By creating agents in MANA and varying these four types of parameters we can 
examine a wide range of behaviors. Of particular interest are the levels of several 
parameters or factors that potentially lead to interesting and/or unexpected results. 
Additionally, the way agents behave with one another, towards the enemy, with civilians 


and the VTUAV could also produce interesting insight into the scenario. 


Once the scenario has been created in MANA and data have been generated, we 
can apply regression and other data analysis techniques to gain a better understanding of 
the interactions between the factors and provide a glimpse of the potential trade space for 


decision makers. 


THIS PAGE INTENTIONALLY LEFT BLANK 


IH. MODEL DESCRIPTION AND DEVELOPMENT 


A. SOME DOCTRINAL PRINCIPLES OF MOUT 


The study of modern urban battles has resulted in some key insights into the 


major factors of urban combat. MCWP 3-35.3 [April, 1998] lists a few of these insights 


as the following: 


MOUT is infantry intensive; 


A tactical battle may have far-reaching implications, with operational and 
possibly strategic repercussions; 


Intelligence is imperative to success; 
Surprise is a combat multiplier; 


As force ratio increases in favor of the attacker, combat duration 
decreases; 


Urban warfare is time consuming; and 


Attack of an urban area is costly to the attacker in terms of resources and 
casualties. 


Because of the nature of urban terrain, units often become separated and isolated. 


This forces the fighting to become a series of small-unit actions. When fighting 


unconventional forces in MOUT, our forces will usually be operating under restrictive 


rules of engagement (ROE). Insurgent forces will exploit our ROE and use the local 


population to their advantage when developing their defensive plan. 


The preferred method of advancing along city streets is the double column. This 


allows 360 degrees of security and mutual support as shown in Figure 1. 
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Figure 1. —_ Patrolling Along an Urban Street [from MCWP 3-35.3, April 1998]. 


During a patrol, enemy contact is possible even if an area is deemed secure. If an 


enemy is engaged, the patrol’s preferred choice should be to return fire immediately and 


conduct a hasty clearing of an adjacent structure in order to seek cover. This allows the 
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Marines to better acquire the enemy’s position and to deliver well-aimed shots in order to 


use their firepower more effectively. 


The nature of combat in urban areas affects the employment of weapon systems. 
Engagement areas are usually close in MOUT. According to MCWP 3-31, 

Studies and historical analyses have shown that only 5 percent of all 

targets are more than 100 meters away. About 90 percent of all targets 

are located 50 meters or less from the identifying Marine. Few personnel 


targets will be visible beyond 50 meters, and most occur at 35 meters are 
less. 


The M16 rifle and M249 Squad Automatic Weapon (SAW) are the most frequently used 
weapons in urban areas primarily because these make up the chief firepower of the 
infantry platoon, but also because strict ROE and the fear of collateral damage prevent 


the use of indirect fire. 


B. SMALL UNIT URBAN OPERATIONS SCENARIO 

This analysis focuses on an infantry platoon conducting patrolling operations in 
an urban environment. The situation is modeled off of the current situation our forces are 
encountering in Iraqi urban areas. The mission of the patrol is to neutralize insurgents 
who are operating in the area, as well as to provide a presence to the local populace. The 
platoon consists of 39 members armed with small arms. They are supported by one UAV 
providing intelligence updates on possible enemy locations that is also capable of 
providing observational cover if the enemy engages the platoon. The situation in the city 
is tense and the citizens are not happy with the presence of U.S. troops. Many citizens 
are sympathetic to the insurgents and will aid them in certain situations. By using the 
urban terrain to their advantage, the insurgents operate in small units and avoid large- 
scale engagements. Using the cover and concealment afforded to them by the buildings, 
they will usually only initiate the engagement when they have the advantage of surprise. 
The insurgents operate as loosely organized irregular units with the ability to blend in 
with the local population, but because of their desire to make a political statement and 


influence the local citizens, they also have a desire to make their presence known. 
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C. MANA 


All models are wrong, but some are useful. 
George Box 


As explained in Chapter I, we chose MANA as the platform for exploring this 
scenario. Much of the information that follows is taken from the April 2004 version 
3.0.35 user’s manual [Galligan et al., 2004]. MANA was developed because of a 
frustration with other more complex, physics based models that were available. While 
highly detailed, these models were unable to capture other intangible factors of combat— 
such as human behavior—which can be critical determinants of battle outcomes. The 
developers offer this advice for those using MANA: 

There must be a clear idea of which aspect of warfare the scenario is 

addressing, and what the entities are trying to do. Though such an 

approach may seem pre-potted, the non-linear nature of the model ensures 

that, regardless of the modeller’s preconception, a_ startlingly large 

number of outcomes are possible. Such a range of outcomes is 


characteristic of complex adaptive systems, and occurs even with quite 
simple rules of behaviour [Galligan et al, 2004]. 


It is this range of possible outcomes that is of interest to decision makers. With MANA’s 
user’s manual and Graphical User Interface (GUI) it is relatively simple to build a 


scenario in a short amount of time. 


D. MODELING OUR SCENARIO IN MANA 
This section focuses on how our scenario’s battlefield and agents were created in 
MANA. Screen shots are shown to aid the reader in understanding model development 


process. 


1. Battlefield Development 

The MANA battlefield is made up of grids that can range from 50 to 1000 pixels 
(or cells) on a side. The default terrain is a 200 x 200 grid. Our model uses a 1000 x 
1000 design, which allows for the highest resolution. Each cell can be occupied by a 
single live entity. The map file is based on a standard Windows bitmap and different 
colors have an affect on an agents’ ability to shoot, move, or communicate. MANA has 


preset terrains for its default battlefield, but the user can create additional terrains to 


12 


provide cover and concealment, impede or facilitate movement and communication, or 
act as barriers. Figure 2 shows the Scenario Map Editor where changes can be made to 


the terrain. 
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Figure 2. Scenario Editor. 


2. Creating MANA Agents 

In MANA a group of agents of any size (1-1000) is known as a squad. These 
squads have the same properties and can change between states as a group or 
individually. States are a set of values that determine the agents’ behavior. This ability 
of agents to change their behavior based on a stimulus is a key component to modeling 


combat behavior. The squads created for this scenario are listed in Table 1. 
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Table 1. | Urban Patrol Squads 


Squads can be guided through the scenario by a series of waypoints designated by the 
user. Their propensity to go to a waypoint is determined by the user via the personality 


settings of each squad, which we describe shortly. 


The squad’s organic situational awareness (SA) and inorganic SA are important 
aspects of modeling information and communication on the battlefield. Whenever a 
squad member sees or detects another entity on the battlefield, this detection is 
transmitted to other members of the squad and stored in memory for a user-specified 


number of time steps. This allows the squad to have a collective picture of the battlefield 
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and make decisions based on their personality weightings. Important to this scenario are 
the squads’ inorganic SA. Inorganic SA is information passed to one squad by another 
squad. Like the squad’s organic SA, the inorganic SA is stored in memory for a user- 
specified number of time steps. This feature allows the user to determine the importance 
of the age of information when making decisions. In the Inorganic SA Tab (Figure 3) we 
see that the user has the ability to vary certain factors that pertain to the communication 
Accuracy, reliability, and 


aspects of passing information over a communication net. 


latency are just of few of the parameters than can be varied. 
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Figure 3. Inorganic SA Communication Properties. 


A squad’s current organic (internally generated) or inorganic SA can be viewed at 
anytime by selecting the appropriate SA from the view menu, as seen in Figure 4. Figure 
4 displays the information being passed to the squad via the communications link of other 
designated squads. In our scenario the inorganic SA that the blue squad is receiving is 
from the UAV. Different colored boxes are displayed around different red icons. The 
yellow boxes indicate the UAV has detected a hostile civilian and the grey boxes indicate 


an insurgent detection. This information is passed to Squad 1 and displayed in Figure 4. 
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Figure 4. | Squad Inorganic SA (Best Viewed in Color). 


3 The Urban Patrol Scenario 

Figure 5 displays a screenshot of the initial condition of the scenario for this 
analysis. The terrain is an urban environment with buildings, roads, alleys and structures. 
This map is actually modeled off of the MOUT training facility at Ft. Polk, Louisiana. 
The colors of the map represent various structures and each has unique cover and 
concealment attributes as well as an effect on squad movement. Yellow represents a 
major street. It offers ease of movement but no cover and concealment. Grey areas are 
walls or other solid structures. These areas offer good cover but entities are unable to 


pass through or over them. The light green areas represent the interior of buildings. 
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These areas offer good concealment and restrict movement. The black area of the map is 
representative of off-road urban terrain such as alleys, sidewalks and small streets. These 
areas offer little cover and concealment, while movement is only slightly more restrictive 


than that of major streets. 
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Figure 5. | Urban Patrol Scenario (Best Viewed in Color). 


The blue patrol starts at the bottom center of the map and proceeds north through 
the city following a series of waypoints. The blue platoon consists of two squads totaling 
39 agents. The blue UAV will provide intelligence to the patrol by conducting a search 


of the city utilizing an alternating lawnmower search pattern. The UAV can be 
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dynamically re-tasked, represented by a state change, to provide observation cover should 
the patrol encounter enemy agents along the patrol route. Following doctrine, the two 
squads move in parallel to one another along the outer edges of the street. Barring any 
enemy contact the platoon has a propensity to simply move from waypoint to waypoint. 
Once the squad makes contact with the enemy it will first seek cover and then proceed to 
engage the red agents, again following MOUT doctrine. If the patrol is not engaged with 
the enemy and the UAV detects and transmits enemy contact to the patrol, then the patrol 


will have a propensity to move towards the UAV contact to engage. 


There are ten squads of insurgents, represented on the map as red soldiers: nine 
squads of three agents each, and one squad with one agent. The insurgents have a 
tendency to remain inside buildings, where they are difficult for the UAV to detect, until 
they see the blue patrol. Insurgents occupying the interior of a building or engaging blue 
forces from windows have a lower probability of detection than those engaging in the 
open. This lower probability of detection is represented in MANA by higher stealth 
settings for the red agents. Once the red agents spot the blue patrol, they then begin to 
move and engage, resulting in decreased stealth and increased probability of detection. 
The enemy IED is represented by a red cross. An observer, represented by a red soldier 
in the prone position, controls the IED. If this observer notices that blue soldiers are 
within range of the IED, he will arm the device. If the observer is killed prior to arming, 
the device is disabled. The IED can only be armed if the observer has a clear line of site 
to the device. In order to represent randomness in the IED’s reliability, the observer’s 
position is randomly generated so that it will not have a clear line of site to the IED for 


each run. 


Neutrals are represented as yellow figures on the map. Their initial placements 
and movements are both random. If they come in close contact with an insurgent, they 
may undergo a state change and become hostile to the blue forces for a period of time. 
This state change is represented on the map by the figure changing colors from yellow to 
red. 

4. Squad Personalities 

The Squad Personalities screen, shown in Figure 6, is the method MANA uses for 


establishing an agent’s characteristics for certain types of actions. The various 
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weightings given under agent, squad or inorganic SA affect the squads’ propensity 
towards certain actions. These ordinal weightings can vary from —100 to 100. For 
example, an agent with a weighting of 100 for Enemies will have a strong desire to move 
toward enemy agents, while an agent with a weighting of —100 will have a strong desire 
to move away from enemy agents. It is worth mentioning that it is not necessary for an 
entity to have a weighting of 100 to have a strong desire to move toward the enemy. As 
long as the Enemies weighting is positive and greater than any of the other personality 
weighting, the entity’s strongest desire will be to move toward enemy agents. In Figure 6 
we see the personalities of the blue platoon in its default state (indicated by the 
highlighted box at the top of the Trigger State window shown on the right-hand-side of 
the screen). The greatest influence on the agents is the detection of an enemy agent via 
the blue agent or blue squad organic SA. This propensity to engage the red agents is in 
keeping with the mission of the patrol. Because the weighting of the squad SA is greater 
than inorganic SA, the squad will not break contact with an enemy to seek other enemy 
reported elsewhere by the UAV. If the patrol is not engaged and receives inorganic SA 


from the UAV, it will have a propensity to move towards the inorganic enemy contact. 
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Figure 6. Personality Settings of Squad 1 in Default State. 


Checking a trigger state in the Trigger State window allows the user to change the 
personality weightings, or other parameters in other tabs, of the agents based on certain 
actions of other agents or how an agent is passed information. The trigger state is given a 
user defined duration and fall-back state once the specified time in the trigger state has 
elapsed. Both these are shown at the bottom of the Trigger State window. For example, 
the state change in Figure 7 is for the blue patrol. Upon enemy contact, the patrol will 
have a propensity to seek cover for a duration of 100 time steps, and then return to its 
default personality. This represents the doctrinal tactic of seeking cover, returning fire, 


and then proceeding to clear a building. 
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Figure 7. Duration and Fallback for the Squad En Contact Trigger State. 


5. Ranges Tab 
Figure 8 displays the Ranges Tab for the blue platoon. The characteristics in this 
tab can change based on the state of the squad. In the General window we can designate 


an icon for the squad, give it an allegiance, assign it a threat for the enemy’s SA, give it 


21 


an agent class so that the enemy can determine what weapon to use against it, and choose 
the squad’s movement speed. Our blue platoon, like other foot-mobile agents, has a 


movement speed of 1, which means it can move one pixel in one time step. 
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Figure 8. | Squad Ranges Tab. 


The Enemy Interaction window designates how many hits it will take to kill the 
agent, the agent’s stealth, and the armor thickness. The stealth value is the percent of 
time (0-100) that the agent will not be detectable. For our scenario, the blue’s stealth is 
very low due to its desire to show a presence. The red stealth is much higher but will be 


reduced if it engages the blue forces. 


The Misc window designates how close an entity must be to a waypoint in order 
to consider it reached. A value of 0 means that each entity would have to be exactly on 
top of the waypoint in order to consider it reached. This would result in the squad being 
channeled into a particular area, which would decrease dispersion and increase the 


chances of casualties. For the purpose of our scenario, the agent must be within 50 
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meters to consider the waypoint reached. The Sensor Capabilities window for an agent is 
used to specify the range over which it can detect and classify another agent. 
Classification ranges can be further broken down into probabilities at different ranges. 
For our scenario, we have locked the detection range to the classification range and set 
the range to 50 for our blue platoon. This distance is in keeping with MCWP 3.35.3. The 
Fuel window is used for agents needing refueling. It was not used in this scenario. 

6. Weapons Tab 

The Weapons Tab is shown in Figure 9. The Weapons Tab allows the user to 
define how many weapons the agents can use, as well as the weapon types and 
accuracies. In this scenario the blue agents have greater kill probabilities than red agents 
to reflect better weapons and training. The weapon type used by all agents, other than the 
IED, is a kinetic energy or small arms weapon. Blue agents also have a second weapon 
type that allows them to engage targets using the inorganic SA provided to them from the 
UAV. Because our forces will normally be operating under strict ROE in this type of 
scenario, the weapon used based on inorganic SA is also a small arms direct fire weapon, 
rather than a high explosive round, such as artillery or grenade launchers. The IED is a 
high explosive devise with a kill radius. Ranges for all the weapons in this scenario are 


based on MCWP 3-35.3. 
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Figure 9. | Weapons Tab. 


7. Movement 

Figure 10 depicts the Algorithm Tab that controls the movements of the agents. 
The Stephen Algorithm is the default setting and is used for this scenario. This algorithm 
considers all moves within the range of the agent, and then chooses a move based on the 
agent’s personality weighting and movement constraints. The Precision Move Selection 
determines the randomness in an agent’s movement. A low setting corresponds to little 
randomness, which is the setting for blue agents in our scenario. Red agents have a 


higher setting, giving them a greater randomness in their movement. 
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Figure 10. Movement Algorithm Tab. 





Under General Movement Settings the user can further define movements. For 
instance, if the Going affects speed option is checked, then the type of terrain will affect 
the agent’s speed and ability to navigate obstacles. This option is checked for all agents 


in our scenario except for the UAV, which flies over structures on the ground. 


This concludes our discussion of model development. We have seen that MANA 
is a simple tool that allows the user to create detailed scenarios in a fairly short amount of 
time. The model developed was specifically for a small unit conducting patrolling 
operations in an urban environment with a UAV. From the discussion it is not difficult to 
imagine countless different scenarios with different parameters that could be examined 


using this tool. 
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Hl. ANALYSIS METHODOLOGY 


The Data Farming environment of MANA is excellent for exploring the effects 
and interactions of many factors over many levels. MANA coupled with high 
performance computing provides decision makers with potentially interesting outcomes 
based on the interactions of the various factors being modeled. With many factors being 
examined over many levels, it is essential that experimental designs be chosen that can 


offer an intelligent way to sample over a wide range of factors and levels. 


This chapter explains the methodology for analyzing the urban patrol scenario. 
We will discuss the robust design process that will be used for designing and analyzing 
our experiment. We will discuss the measures of effectiveness (MOEs) that were chosen 
for this scenario and the statistical methods for analyzing the data and factors important 
to the success of the model. Also covered in this chapter is the data collection plan 


chosen for exploring our MANA scenario. 


A. ROBUST DESIGN 

The process used to determine the best design for this scenario is known as robust 
design. This approach is an optimization process that espouses that a design’s mean 
performance should not be the only criteria for the best design—a good system design 
should also be relatively insensitive to uncontrollable factors in the environment. 
Genichi Taguchi, who used this process in the manufacturing field for engineering 
product design, pioneered this approach. 

He found that it was often more costly to control causes of manufacturing 

variation than to make a process insensitive to these variations, and 

through the use of simple experimental designs and loss functions was 


often able to greatly improve product performance by ‘building in’ the 
quality. [Sanchez et al., 1996] 


Although originally used in the manufacturing field, a design insensitive to 
uncontrollable factors is particularly appealing for military systems. In the combat 
environment, military systems are constantly affected by uncontrollable sources of 
variation. The framework for a robust design that follows is detailed in [Sanchez et al., 


1996]: (1) select the performance measures; (2) specify a loss function; (3) identify the 


P| 


factors; (4) plan the experiment; (6) analyze the results; (7) refine the metamodels; (8) 


select the best process design. In the following sections we examine each step. 


B. MEASURES OF EFFECTIVENESS (MOE) 

An MOE is a quantitative measure of performance. In our scenario we have 
chosen the red:blue kill ratio, also known as the exchange ratio (ER). Rather than just 
look at red kills, we would like a performance measure that takes into account blue 
casualties. Although we expect blue casualty rates to be high in this scenario, we would 
like to find a combination of factors that would make the ratio as high as possible. For a 
robust design it is helpful to have an associated target value t, which represents the ideal 
exchange ratio. In a perfect world there would be no blue casualties; however, we know 
that this is virtually impossible in urban warfare. Actual exchange ratios have varied 
widely in urban warfare. For instance, the exchange ratio for the battle of Hue City 
during the Vietnam War was 1.6, while the exchange ratio for the attacking Iraqis during 
the battle for Khorrasmshahr during the Iraq/Iran War was 0.76 [Lamont, 1999]. Because 
the battle for Hue was seen as a tactical victory we will use the optimistic value of tT = 1.5 


as our target. 


C. THE LOSS FUNCTION 

The next step in the robust design process is to define a loss function that will 
convert the performance measure into dimensionless units. This function measures the 
cost associated with the variation of the system relative to t. If the system was ideal, our 
MOE would be equal to tT and the variance would be zero. However, this is highly 
unlikely to occur, so some way of defining and measuring loss is necessary. For our 
design we will utilize a quadratic loss function. Letting x and Y(x) represent a vector of 
decision factors and the associated MOE, respectively, and letting Q be the noise factors, 


then the quadratic loss function can be written as: 
I(x) =cL¥ (x) - 1] (1) 


where the scaling constant c is used to convert losses into monetary units. For our 
purposes the scaling constant will be one. From equation (1) the expected loss is defined 


as: 
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E(l(x)) =clo, +(u. - 9] (2) 


where 0% is the variance and pw, is the mean associated with the vector of decision 


factors x; see [Sanchez et al. 1996] for details. Because the variability is not constant 
across different designs, the loss function in equation (2) is a better measure of a ‘good’ 
system than just considering the difference between the mean of the MOE and the target 


value. 


Although we are using a quadratic loss function for this study, it is by no means 
the only loss function appropriate for designing robust systems. We have chosen the 
quadratic loss functions for several reasons. The first reason is that the values are easily 
interpreted to real world terms—the square root of the loss, like the standard deviation, is 
in the original units of the problem. Second, we know from examining our data that we 
will not encounter values with the means greater than 1.5. This is important because, 
assuming that high exchange ratios are better, the quadratic loss function would associate 
large losses with any exchange ratios much higher than our target value. With this 


potential problem in mind, another possible loss function is the following: 
1(x) =—log(Y (x)) =—log(exchange ratio) (3) 


The expected loss can then be estimated by the average of the —log(exchange ratio) 
values for all runs and replications at design point x. With this loss function we are not 
penalized for achieving exchange ratios higher than a specified target value—this loss 
function does not even require the analyst to specify a target value. One disadvantage of 
using —log(exchange ratio) is that all observed exchange ratios must be non-zero. It is 
also not as natural a scale for people to use to compare and contrast alternatives. In our 
analysis we will consider the results of the loss function in equation (1) with a target 
value of 1.5, show what happens if a slightly different target value is used, and compare 
with those using the loss function in equation (2) in order to ensure that our results are 
consistent across both loss functions. Further work could be done involving more general 
loss functions, or calibrating the loss functions to reflect the decision-maker’s assessment 


of relative risk, but that is beyond the scope of this study. 
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D. FACTORS AND SETTINGS 
The next step in the design process is to identify factors that might contribute to 


the MOE. The factors are classified as decision factors and noise factors. 


Decision factors are controllable in the real-world setting that is being modeled. 
The noise factors are those factors that are not controllable, or are difficult or costly to 
control. Because the primary goal of this thesis is focused on the VTUAV, those 
decision factors associated with the VTUAV will be of primary interest. Based on some 
exploratory runs in MANA, we chose to use the seven decision factors and three noise 


factors in Table 2 in the final production runs. 





DECISION FACTORS 








VTUAV Default Sweep Width 

Observation Cover 
Speed 300 1000 
Latency 0 100 
Comm Reliability 5 a5 
Information Accuracy 5 95 
Blue Platoon | Default Aggressiveness -5 30 
































NOISE FACTORS 













Low 
Squad Name | State | Factor Name | Setting 
Civilian Default Red Influence 
Red Enemy Contact | Aggressiveness 
Default Stealth 
































Table 2. Decision Factors and Noise Factors 


In Table 2, the decision factors and noise factors are outlined by squad name, state 
of the agent, the factor name, and the low and high settings. The sweep width of the 
VTUAV is related to the field of view. Because the battle field in MANA has been 
defined as | meter equal to 1 pixel, the conversion is straightforward for the sweep width. 
The settings range from 20 to 250 meters. The observation cover factor is the VTUAV’s 
propensity to provide observation cover to the blue platoon upon enemy contact. Speed 


in MANA is relative to the slowest entity on the battlefield. In our scenario the red and 
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blue infantry both have a speed of 5 mph. The VTUAV’s speed ranges between 13 
nautical miles per hour and 44 nautical miles per hour. These speeds are relatively slow, 
but consistent with urban surveillance. Latency is the delay in communication between 
the VTUAV and the blue platoon. The settings are based on time steps. For example, a 
value of 100 translates into a 100 time step delay in information reaching the blue 
platoon. Communication reliability is the likelihood that a message will be successfully 
delivered to the receiving squad. Information accuracy is the probability that a contact’s 
type will be passed correctly to the receiving squad. The final factor is blue 
aggressiveness. This controls the blue platoon’s propensity to move toward enemy 


contacts. 


The noise factors chosen are red influence on the civilians, which affects civilian 
hostility towards blue forces, red aggressiveness, and red stealth. A civilian’s hostility is 
affected by nearby red agents. Upon contact with a red agent, a civilian becomes hostile 
towards blue forces for a number of time steps, and then reverts back to their neutral 
state. This noise factor has the potential to be somewhat controllable through various 
pacification efforts not explored in this study. For the purposes of our investigation we 
use a range of potential hostility to include civilian hostility as a noise factor. Red 


aggressiveness and stealth are the final two noise factors for our study. 


E. EXPERIMENTAL DESIGN 

After choosing the factors and levels we must now focus on the data collection 
plan. For our study we will use a crossed decision factor x noise factor plan. This 
approach uses the same plan for each of the noise factors, which allows us to examine the 


variability across the noise space [Sanchez et al., 1996]. 


Utilizing a full factorial design for our study would not be practical given the 
levels we wish to examine and expected non-linear response. For example, if we were to 
look at our 10 factors each having only 4 levels, we would need 4'° design points. This 
would require 1,048,576 runs to obtain a single piece of data for each design point. If it 
took only one minute to run the simulation it would require almost 2 years for a single 


data point per design point. Replicating the design to gather information about the 
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distribution of results for each design point would further increase the time required for 
data collection. Obviously this approach is not practical for even a modest number of 


factors with many levels. 


Given the factors and levels we wish to examine, we chose a Latin Hypercube 
Design (LHC). The LHC designs are excellent for space filling and used to efficiently 
look at many factors with many levels when concerned about extreme non-linearity 
[Lucas et al., 2002]. This is done by not only sampling on the edges but also in the 
interior of the design area. For our study we will use a Nearly Orthogonal LHC 
(NOLHC) design. This design provides us with two important characteristics: 
orthogonality and good space filling [Cioppa, 2002]. Orthogonality is important in order 
to have low correlation, reducing the effects of multicollinearity on the model. Good 
space-filling is important because the design points are scattered throughout the 
experimental region making it easier to detect any nonlinearities. In order to construct a 
NOLHC for this study, a Microsoft Excel Spreadsheet (implementing Cioppa’s designs) 
constructed by Professor Susan Sanchez was used to generate the design points [Sanchez, 
2004]. Figure 11 displays the spreadsheet for a 1-7 factor design. The user can pick an 
orthogonal or nearly orthogonal design based on the number of factors to be examined. 
The minimum and maximum levels and names for each factor are then entered into the 
appropriate cells. The default factor levels (given as the integers 1-17 in Figure 11) are 
appropriately converted. Each row then specifies the combination of factor levels for a 
single design point. The ranges used in our investigation mean that some rounding of the 


factor levels occurs, so the resulting design is not quite orthogonal. 


32 


low level 
__ high level 
factor name 














Figure 11. LHC Design Spreadsheet (from: Sanchez, 2004). 


In order to better utilize the space filling properties of the LHC design, we linked 
two 7 factor OLH designs together using an algorithm described by Cioppa [2002, p. 54]. 
The resulting scatter plot and correlations for each pair of factors are shown in Figures 12 
and 13 respectively. Each region is reasonably filled with points, so our experiment will 
provide insight for intermediate values of the factors, not just the extremes. When linked 
together, we have an experimental design with 33 configurations. Crossing this design 
with a similar design for the noise factors gives us 1089 design points (33 x 33). For 


each of the design points we will conduct 40 replications for a total of 43,560 runs. 
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Figure 12. 


Scatter Plot. 


Figure 12 shows that our design has reasonable space-filling behavior. Table 3 


shows that despite the rounding necessary to obtain the factor levels for our experiment, 


all pairwise correlations are still very near zero. 





Sweep Width 
Observation Cover 
UAV Speed 

UAV Latency 

UAV CommReliability 
UAV InfoAccuracy 
BlueAggresiveness 





UAV UAV UAV 
Sweep Observation UAV UAV Comm Info Blue 
Width Cover Speed Latency Reliability Accuracy Aggressiveness 
1 -0.0007 -0) -0.0017 0.0019 0 -0.0086 
-0.0007 1 0 -0.0079 0.009 -0.0018 0.0044 
-0.0014 0 1. 0.0007 0.0033 0 0.0069 
-0.0017 -0.0079 TE-04 1 0.0031, 0.0024 -0.0038 
0.0019 0.009 0.003, 0.0031 1 -0.0013 -0.0045 
0 -0.0018 0 0.0024 -0.0013 1 0.0059 
-0.0086 0.0044 0.007 -0.0038 -0.0045) 0.0059 1 








Table 3. Correlation Matrix 
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F. CONDUCT THE EXPERIMENT 

With the scenario complete and design points determined, the factors are now 
ready to be farmed. The MITRE Corporation, in support of the Marine Corps’ Project 
Albert, maintains a supercomputing cluster that supports data farming projects. After 
emailing the MITRE support team an XML file containing the base scenario, a separate 
design file listing the factors to be examined and the corresponding factor settings for 
each design point, and the number of replications, a script file is created that varies the 
factor levels according to the user-specified design. Output files are returned in comma 


separated value (.csv) format [Hakola, 2004]. 


After running 40 replications of the experiment for each design point, we collect 


the mean and variance for each point. Letting Y be the random function of the decision 
factors {X,} and the noise factors {W,}, we have the following equations for the mean 
response Y, and mean variance V, corresponding to the decision factor configuration i 


and noise configuration j: 


— 1 Ny — 
Yee) ¥5 (4) 
n, j=l 
-_ lL Ss syvu,leo 
Vi =——) (Vi -Y:) +—)'s; (5) 
N,, —l j=l n,, j=l 


where n,, is the number of noise factor points [Sanchez et al., 1996]. The total variance 


of the design point in equation (2) is the sum of the inherent variance of the simulation 


run and the extrinsic variance across the noise space. 


G. ANALYZE THE RESULTS 

After running the model and collecting the mean and variance for each design 
point, we use regression to develop metamodels for the mean and logarithm of the 
variance. Regression analysis is used to take advantage of the relationship between two 
or more variables so that we can gain information about one of the variables by knowing 
the values of the others [Devore, 2000]. Our metamodels are functions of the decision 


factors taken over the noise space. We use a design for fitting main effects, interactions, 


GN, 


and quadratic effects. The variance is transformed logarithmically for stability, and the 


initial models will be of the form [Sanchez et al., 1996]: 


<p 


u=Y, =B+BX, + -+£8.X, +B X,X,+--+B,X, 4X, + quadratic (6) 


In(o?) = In(V) = 7, +9,X, +--+ FX, + FX Xp t+ FX, X, + quadratic (7) 


The data analysis software package used for this thesis was JMP Statistical Discovery 


Software ™, 


H. REFINE THE METAMODELS 

The initial models will be evaluated based on their coefficient of determination 
(R*) value. This value is the proportion of variability in the response that is explained by 
the model. Clearly, R* alone may not be the best measure of a ‘good’ model. With a 
full model of main effects, interactions, and second order polynomials it is possible to 
have an extremely high R* but few degrees of freedom for error. It would also be 
difficult to explain what is going on with a model of this many terms, rendering it 
virtually useless. The initial model should be assessed and any insignificant terms 
removed. The test used to determine which individual terms will remain in the model is 
the t-test. The equation for the t-statistic is the following [Hamilton, 1992]: 


b, = B, 
SE, 


T= 


(8) 


where b, is the coefficient estimate for factor k, {, is the hypothesized value of the 
coefficient for factor k , and SE, is the standard error of b,. The associated p-value of 
the t-statistic will determine the significance of the term. Reasonable levels of 


significance are usually defined as any p-value less than 0.05 or 0.01. For our study we 


will use a value of 0.05. 


I. SELECT THE ‘BEST’ DESIGN 
We can use the loss function to examine the metamodels to determine the most 
desirable decision factor values for our scenario. These values may be different than 


those we would suggest if we were basing our decision on mean performance alone. 
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This chapter has outlined the robust design methodology and techniques for 
analyzing the data of our scenario. Chapter IV will present a detailed analysis based on 


this methodology. 
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IV. DATA ANALYSIS 


This chapter details the analysis and findings. We proceed using the methodology 
described in the previous chapter. We will begin by examining the data and giving an 
initial assessment with graphical analysis and regression trees. We will then analyze the 
data using multiple regression models with main, interaction, and quadratic terms. Next, 
we will refine the models by including only the relevant terms. Using the refined models 


with the loss equation detailed in Chapter II], we will select the best design. 


A. INITIAL ASSESSMENT 

Upon receipt of the data, we examined the data to make certain there were no 
problems with the data quality. We consolidated the runs of each of the 1089 design 
points and produced our mean exchange ratios. The distribution plot of the data is shown 


in Figure 13. 





Mean(Exhange Ratio) 
' Moments 
Mean 0.9082214 
Std Dev 0.1121232 


Std Err Mean 0.0033977 
upper 95% Mean 0.9148882 
lower 95% Mean 0.9015547 
N 1089 














Figure 13. Distribution Plot of Mean(Exchange Ratio). 


We see from the distribution plot, as expected, all the exchange ratios are positive. 
Also as mentioned in Chapter III, all mean exchange ratios are less than our target value 
of 1.5, so using a target value of 1.5 provides a motivation toward continual 
improvement. The distribution has positive skewness, including some high values that 


would be considered outliers if the data followed a normal distribution. These outliers 
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correspond to good outcomes, but keep in mind that in a robust design framework the 
best decision might not produce the same outcome as a decision based on mean 


performance alone. 


After examining the data we then developed a regression tree, which can give 
insights to the possible important decision factors and noise factors that drive our model. 
A regression tree partitions the data recursively according to the relationship between the 
decision factors and the response [JMP® Statistics and Graphics Guide, 2002], but does 
not assume a specific structure for the form of this relationship. Because the focus of this 
study is developing a robust model, we will begin by developing a regression tree using 
only the decision factors. Later in the chapter, we will build a regression tree using both 
the decision and noise factors to see if we can gain additional insights into the 
development of the appropriate tactics in various settings. Figure 14 is a regression tree 


considering only the decision factors. 
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Sweep Width < 49 
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Mean = .88 











Figure 14. Regression Tree, Decision Factors Only. 


With 33 design points represented and the total mean shown in the top box, we 
have an R’ of .89 after only 4 splits of the data. The box at the bottom left of the tree 
(colored red) represents the least favorable conditions, yielding an average exchange ratio 


of 0.85. The box at the bottom right of the tree (colored green) represents more favorable 
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conditions, yielding an average exchange ratio of 0.99. (In this instance we show only 
one red and one green leaf, but in general there could be several leaves with similar 
values.) For instance, and not surprisingly, the best exchange ratios are achieved with 
high communication reliability, high accuracy, and a high sweep width. It is interesting 


to note that if the sweep width is low a higher accuracy does not improve the mean. 


With these initial insights into the data we will now begin to develop our 


regression models for a more detailed look at our decision factors. 


B. DEVELOPMENT OF THE MODELS 
As outlined in the previous chapter, we use regression to develop metamodels for 
the mean exchange ratio and the logarithm of the variance as functions of the decision 


factors. 


1. Mean Exchange Ratio Main Effects 

The regression model that we are developing associates the average response 
(calculated over the noise space) to the decision factors. We began by using only main 
effects to get an idea of the impact of these factors alone on our model. We developed 
the main-effects model using the mixed stepwise function in JMP. This function uses 
alternating forward and backward steps, allowing terms to enter the model on the forward 
step and leave the model on the backward step, based on a significance level for each. 
For our model we allowed terms to enter the model at a significance of .25 and removed 


terms with significance less than .05. 


The main-effects model that resulted consisted of three terms with an R? = .78, 
indicating that 78% of the variability in the exchange ratio is explained by these three 
terms (p-value < .0001). Reliability, accuracy, and aggressiveness were found to be 
significant at the .05 level and the plot of the residuals vs. the predicted values is shown 
in Figure 15. The solid red line corresponds to predictions from the regression equation, 
while the dashed red lines represent a 95% confidence band for the mean predicted 
values. Ideally, the points should be scattered around the prediction line with no apparent 
pattern. We see from the graph that there is a group of points that fall below the lower 
confidence limit. While the R’ and the graph look reasonably good, those points indicate 


that we might be able to do better. 
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Figure 15. Actual vs. Predicted Exchange Ratio. 


Z Final Model Mean Exchange Ratio 
Again using the stepwise function in JMP with the same significance criteria in 
the main effects model, we developed the final model. The final model consisted of six 
terms: four main effects, one interaction, and one quadratic with an overall p-value < 
0001. Looking at the actual versus predicted exchange ratio in Figure 16 we see that the 
group of troublesome points in Figure 15 has decreased. There does not appear to be any 
pattern in this plot, indicating the assumption of independent and identically distributed 
errors is reasonable. The R’ for this model is .91, giving us better predictability than just 
the main-effects model while still having relatively few terms. The final model of the 
exchange ratio is represented by the following equation: 
ExhangeRatio = 0.7598 +0.00007(Sweep _ Width) + 0.00157(Reliability) 
+0.00122( Accuracy) +0.00149( Blue __ Aggressiveness) 


+0.000022| (Reliability —50.0606) * (Accuracy —50.0606)] 


—0.000004{ (Sweep _ Width -135.061)* (Sweep _Width—135.061)] (8) 


The interaction and quadratic term have the mean point subtracted from the factor value 


to ensure proper scaling. 
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Figure 16. Actual and Residual vs. Predicted Exchange Ratio. 


Summary statistics for the final model can be seen in the Appendix. Of the seven 
decision factors, four appear as main effects in the final model. Although sweep width 
has a p-value of .19 it has been included as a main effect due its significance (p-value < 
.0002) as a quadratic term in the model. Figure 17 displays the leverage plots for the 
significant main effects of the model. The UAV communications reliability has the 


strongest influence, followed by UAV information accuracy and blue aggressiveness, 
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Figure 17. Leverage Plots. 


The model’s interaction term, communication reliability and information 
accuracy, can be seen nicely using the contour plot in Figure 18. The blue regions circled 
in the upper right-hand corner represent areas with higher mean exchange ratios. 
Intuitively it seems reasonable that these two factors would have an interaction effect. 
Perfect communication with imperfect information or vice versa is virtually useless, 
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but—like the regression tree—the contour plot can give us insights into the potential 
trade space between these two factors. This can be especially important in an urban 
environment where potential threats can be difficult to identify and communication 


sporadic. 
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Figure 18. Contour Plot of Accuracy and Reliability vs. Mean Exchange Ratio. 


The contour plot in Figure 19 shows the mean exchange ratio as a function of blue 
aggressiveness vs. information accuracy. As the circled area on this plot shows, we see 
that as the information we receive becomes more accurate the need for higher 
aggressiveness in order to maintain high exchange ratios decreases. This could have 
potential benefits in areas where we want to minimize our aggressive posture in order to 


win the hearts and minds of the local population. 
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Figure 19. Contour Plot of Accuracy and Aggressiveness vs. Mean Exchange Ratio. 


The final term in the model is the quadratic term of sweep width. This term 
implies that there is a point where too large of a field of view is detrimental to the 
exchange ratio. Generally this would seem counterintuitive; the wider area a UAV can 
see the better. However, if we look at this factor in the context of the situation we are 
modeling it seems to make more sense. The area of the patrol is relatively small and the 
capability of a small foot mobile unit to cover large distances in an urban environment is 
limited. With the UAV able to detect more red agents at greater distances, the blue forces 
will move to engage these detections. If the detections are lost by the UAV before the 
blue forces arrive, the blue forces will move back to their original patrol route or to the 
most current UAV detection. Moving over larger distances it is not difficult to imagine 
an accordion effect between more recent detections, resulting in fewer engagements and 
lower exchange ratios. It might be argued that this is an artificiality of the model, but it 
could also be seen as information overload. With too much information the blue force 
becomes indecisive and ineffective. Figure 20 displays the prediction profiler graph from 
JMP. The profiler is a useful tool to see how the predicted values change as you change 
one factor at a time. Figure shows the settings that maximize the mean exchange ratio 
based on the metamodel. Notice the curve for sweep width and the point of decreasing 


effectiveness. This means that moderate sweep values are best. 
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Figure 20. Prediction Profiler with Optimal Settings for Maximum Exchange Ratio. 


3. In(Variance) Model 

The In(Variance) model was developed in the same manner as the mean exchange 
ratio model. We considered all main effects, two-way interactions, and quadratic effects, 
allowing terms to enter the model with significance greater than 0.25 but eventually 
eliminating terms with significance less than .05. Using the stepwise function in JMP we 


obtain the following metamodel equation: 


In(o”) = —2.500952 + 0.00060(Sweep _ Width) + 0.0067 1( Reliability) 

+0.00532( Accuracy) + 0.00619( Blue __ Aggressiveness) 

+0.00003[(Sweep _ Width —135.061) * (Accuracy —50.0606)] (9) 
—0.00002[(Sweep Width —135.061)* (Sweep _ Width —135.061)] 

—0.00013[( Reliability — 50.0606) * (Reliability —50.0606)] 


Summary statistics for the final model of In(variance) can be seen in the 
Appendix. Figure 21 displays the graphs for actual and residual by predicted. We see 
from the graph that the model produces an R° of .85. The regression is statistically 


significant (p-value < .0001). 


46 


Actual by Predicted Plot 


Residual by Predicted Plot 
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Figure 21. Actual and Residual vs. In(Variance) Predicted. 





In order to make use of the maximization function in the profiler we maximized 


the quantity [-In(variance)] in order to get the best settings for minimum variance as seen 
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Figure 22. Prediction Profiler with Optimal Settings for Minimum Variance. 


Looking at the optimal settings for minimum variance, the values are almost all at the 
other end of the spectrum from the optimal settings for the maximum exchange ratio. 
With the settings for reliability and accuracy at 5 the UAV is virtually nonexistent as an 
intelligence gathering asset. The aggressiveness setting at -5 translates into a blue force 
avoiding engagements with the red forces. Essentially, in order to keep the variance at a 
minimum, the blue force should avoid fighting. Of course this is not an option in urban 


combat. 
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In an ideal world we would be able to simultaneously maximize the exchange 
ratio and minimize the variance using the same decision factor settings. Unfortunately 
this is rarely the case. As the profiler plots and Figure 23 demonstrate, for our scenario it 
is impossible. Figure 23 displays the mean exchange ratio vs. the variance. We see that 
as the mean exchange ratio increases so (in general) does the variance. Even though 
there does appear to be a general trend, this is not a direct relationship. There are 
differences in the variance especially at high exchange ratios. This is not surprising 


given that in order to increase the number of red kills, the blue forces will have to engage. 
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Figure 23. Variance vs. Mean (Exchange Ratio). 


We have seen from the profiler that the best mean exchange ratio the model will 
achieve is 1.16; therefore, if we are to have a chance to come close to our target value of 
1.5 we will need increased variability in the loss function. (Note that the increased 
variability also means the exchange ratio may be much lower.) Now that we have 
developed the metamodels for the mean exchange ratio and the variance of the exchange 
ratio, we will now use equations (8) and (9) in order to minimize the loss with respect to 


our target value of 1.5. 
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C. CHOOSING THE ‘BEST’ DESIGN 

Using equations (8) and (9), along with the quadratic loss equation (2), we can 
look for the most robust configuration. Because the data collection plan uses a LHC 
design rather than a full factorial we will use a grid search over the model’s decision 


factors for the minimum loss. 


1. Minimizing the Loss Function 

In order to look at all possible combinations of all levels of the four decision 
factors common to both metamodels would require 65,205,000 combinations. Instead of 
looking at each level we aggregate the intervals to a more manageable amount of 
combinations. Using the factors and levels of Table 4, we are able to look at a grid of 


9,600 combinations. 








Sweep _ Width Accuracy |Reliability | Aggressiveness 
20 5 5 -5 
40 15 15 0 
60 2 2D 5 
80 2D 35 10 
100 45 45 iS 
120 55 JD 20 
140 65 65 25 
160 re) D 30 
180 85 85 

200 95 95 
220 
240 

















Table 4. Loss Factors and Levels 


Table 5 contains the 33 design points with the resulting mean, In(variance) and 
loss for our target value of 1.5. The design point with the lowest loss tested is the T" row 
(boldfaced and highlighted in yellow) with a loss of 0.433. The design point in row 3 
yields the lowest (worst) mean exchange ratio, but the best (lowest) In(variance); its loss 
is 0.547—26% higher than the most robust design point. The losses associated with the 
design points corresponding to the best mean exchange ratio (1.09) and the worst 


In(variance) (-1.20) are .436 and .534, respectively. 
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Table 5. Mean, In(variance), and Loss(1.5) for the 33 Design Points 


Using equation (2) we determined the factor settings associated with minimum 
loss for the target value of 1.5 was a combination that had not been tested: Sweep Width 
= 20, Accuracy = 95, Reliability = 95, and Aggressiveness = 30. The loss associated with 
these settings was .343 which was 21% less than the best of the design points tested. A 
validation experiment was run at this setting to see whether or not the final metamodels 
are still providing a good fit for these factor settings. The validation experiment yielded a 
mean exchange ratio of 1.06, which is within the 95% prediction interval of (1.007, 
1.123). Similarly the variance and loss for the validation experiment, which were .149 
and .339 respectively, fell within the 95% prediction intervals computed from the 
metamodels. This is strong evidence that the model developed from only 33 design 


points is providing good results in other parts of the design space. This is even more 
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impressive if we consider that the smallest full factorial design that could have looked for 
main, quadratic and interaction effects would have needed 2187 design points for the 
decision factors. A full factorial with 33 levels for each decision factor would require 
over 42.6 billion design points. 

2. Insights into the ‘Best’ Design 

With the best design for our model chosen, we then decided to examine the loss 
function over a large range of possible targets. As mentioned earlier, exchange ratios 
have varied a great deal in recent history. We decided to look at target values ranging 
from 1 to 2 with increments of .01 for a total of 101 different target values. Taking the 
minimum loss value for each target yielded the twelve different designs (or factor setting 


combinations) shown in Table 6. 






































Factor Setting Sweep Width | Accuracy | Reliability | Aggressiveness 
1 240 5 S) 30 
2 20 95 95 -5 
3 20 95 95 0 
4 20 95 95 5 
5 20 95 95 10 
6 20 95 95 15 
7 20 95 95 20 
8 20 95 95 25 
9 20 95 95 30 
10 40 95 95 30 
11 60 95 95 30 
12 80 95 95 30 























Table 6. —_ Factor Settings for Minimum Loss for Target Values 1 Through 2 


Figure 24 shows the best factor settings for a particular target. Notice that for a 
target value anywhere from 1.37 to 1.67, factor setting 9 produces the minimum loss. For 
targets less than 1.1 we see the factor settings approach those that produce minimum 
variance. Although not shown in the Figure, a target loss of .8 results in factor settings 
that are equal to those that produce minimum variance. Conversely, as target values rise 


above 1.1, factor settings approach those that maximize the mean. 
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Figure 24. ‘Best’ Factor Settings for Particular Targets. 


As target values rise above the mean of the data, the loss function favors settings 
that maximize the mean and variance. In order to reach the target value we not only must 
try to maximize the exchange ratio, but we must also increase the variability so that we 
have the potential of reaching the target. As stated earlier, in a perfect world we want the 
exchange ratio to be as high as possible, but in order to achieve that goal we must be 


willing to accept a large amount of risk. 


Recall from Chapter III that in an ideal situation our target value would be equal 
to the mean and our variance would be zero. Figure 25 represents that situation in as 
much as the current model can produce it. With our target value at .78 and our variance 
minimized, we see the lowest loss that our model can produce. The settings for this 
target value are: Sweep Width = 240, Accuracy = 5, Reliability = 5 and Aggressiveness = 
-5. These settings are the equivalent to not fighting—which is obviously not a choice in 
our scenario. The target value would not even be worthwhile to consider, but the graph 
does give visual insights about the effect of the target value chosen and the importance of 


variance on the loss function. 
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Figure 25. Minimum Loss vs. Target Value. 


Let us examine the factors in Table 4 a little closer. Notice that for targets 
between | and 1.1, the UAV is essentially turned off and the blue forces are very 
aggressive. At values below the target of 1, blue forces eventually become the least 
aggressive. While these settings might produce the lowest loss of the entire model, it is 
akin to the adage of “nothing ventured, nothing gained.” In turn, if we look at factor 
settings 2 through 9, we see that the only decision factor that is changing is the 
aggressiveness of the blue forces. In other words, the only change in factor settings that 
is required to minimize the loss for target values 1.1 through 1.68 is the blue force’s 
aggressiveness. Once the aggressiveness setting reaches its maximum of 30, the sweep 
width slowly increases to a maximum of 80 for the target value of 2. 

D. AN ALTERNATE LOSS FUNCTION 
Now let us consider the alternate loss function of equation (3) that was discussed 


in Chapter III. By calculating the loss based on —log(exchange ratio) and plotting this 


a 


against the loss of the quadratic function of equation (2) we can determine how closely 
the two sets of results correspond to each other. Figure 26 shows a scatter plot. With an 
R’ of 0.74 and a correlation of 0.93 the alternate loss function is in close agreement with 
the quadratic loss using a target of 1.5. Two design points might be considered outliers, 
since they have higher quadratic losses than others with similar —log(exchange ratio) 


values, but these are not associated with the most robust design points. 
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Figure 26. Loss Function Comparison 


E. ROBUSTNESS REVISITED 

Recall from the beginning of the chapter that we developed a decision tree using 
only the decision factors taken over the noise space. Now let us construct a new decision 
tree using all of the decision and noise factors using loss as our response instead of the 


exchange ratio. We can see from Figure 27 that similar splits occur. 
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Figure 27. Regression Tree, All Factors. 


The regression tree in Figure 27 represents all 1089 design points and considers 
noise and decision factors together. The overall mean is shown in the top box and the 
tree has an R’ of .24. The boxes colored red and green at the bottom of the tree represent 
the less and more favorable conditions, respectively. For instance, according to the 
regression tree, if we were in a situation were the civilian hostilities were high, it would 


be desirable that our communication and accuracy also be high. 


The biggest factor impacting the mean loss is the influence on the civilian 
population by the red forces, or civilian hostility. When the factor setting is greater than 
33 the mean loss is .54, if it is less than 33 the loss falls to .46. Remember the influence 
ranged between 10 and 50, so a value of 33 would represent an area with just above 50% 
hostile civilians. Depending on the level of hostility by the civilian population, 


communication reliability and information accuracy are the second most important 


a, 


factors on the tree. If the patrol is operating in an environment with high civilian 
hostility, greater communication reliability and information accuracy are both needed in 
order to avoid high loss. Alternately if the civilian hostilities are low, high information 


accuracy is the most important contributor to low loss. 


The regression tree also gives us an idea of the potential trade space given that we 
possess certain kinds of intelligence. For instance, looking at the regression tree we see 
that in situations where the local populace is relatively friendly to our forces, we might be 
able to achieve acceptable results even with a UAV that has low communication 


reliability and low accuracy. 


F. SUMMARY 

In this chapter we developed models for the mean exchange ratio, as well as the 
variance of the exchange ratio. The mean exchange ratio model consisted of seven terms 
that explained 91% of the variability, with three of these terms explaining 74% of the 
variability. The In(variance) model consisted of 8 terms explaining 84% of the 
variability. Using these two models and the loss function, we developed insights into the 
best model based on a specific as well as a range of target values. We also developed 


regression trees for an initial and nonparametric understanding of our data set. 


Recall that the goal of this thesis is to provide insights, rather than numbers, so 
our numerical results should be viewed in this light. The final chapter will link the 


observations from the statistical analysis with possible implications for the real world. 
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IV. CONCLUSIONS AND RECOMMENDATIONS 


Recall that the purpose of this thesis was to gain insights to small unit urban 
patrolling with the VTUAV using agent based simulation. We now summarize our 


findings and provide recommendations for follow-on research. 


A. SUMMARY 

The metamodels and designs developed for the agent based simulation of this 
thesis do not translate literally into the real world of modern combat, nor is this the 
purpose of agent based distillations. What we have been able to observe using these 
models gives us insights into the potential tactics, techniques, and capabilities that might 
be important in similar situations. It also provides insights into the possible trade space 
of technology and asset support given the constant battle over limited resources. This 


thesis also provides a template for studying other scenarios using robust design. 


As stated in the ORD for the VTUAV, it must be able to accomplish missions 
ranging from MOOTW to a large intensity conflict. Because it must be able to operate in 
different environments at different levels of conflict, the VTUAV cannot be specifically 
designed for just one type of operation. Given this requirement, this study focused on a 
scenario with a broad range of environmental noise factors and intensity levels in order to 


generate a robust system capable of operating in different environments. 


A platform that has very reliable communication capability and very accurate 
sensing capability is best suited for the environment modeled. These two factors were 
the most dominant in providing favorable results. A UAV with high reliability and 
accuracy needs only a relatively narrow field of view to provide favorable results for 
small units operating in dense urban environments. Speed, as might have been guessed, 
will probably not play an important role for a UAV in an urban environment. These 
factors might suggest possible flexibility when using UAVs in support of urban 
operations. Because of its broad range of required capabilities, the VTUAV will be a 
very capable, expensive and limited asset. If other, less expensive assets can be used, 
cost savings and flexibility could be realized. As mentioned in the introductory chapter, 


the Marine Corps currently possesses the Dragon Eye UAV. Although a slow and rather 
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unsophisticated over-the-hill reconnaissance asset, upgrading its sensing capabilities 
might make the Dragon Eye a low cost alternative to using the VWTUAV in the urban 


environment. 


By introducing noise into the system and using the tool of regression trees, we are 
given an idea about the potential trade space of required capabilities given our knowledge 
of certain conditions. For instance, we saw that the hostility of the civilian population 
plays an important role in urban combat. If civilian hostilities are high, a more 
sophisticated UAV appears to be necessary for the patrol to operate effectively. In 
environments with more friendly populations, the Dragon Eye or other less sophisticated 


intelligence assets might be sufficient to accomplish the mission. 


We see from the metamodels and the loss function analysis that the intelligence 
gathered from the UAV is only a piece of the puzzle. How the blue force uses that 
intelligence is the other piece. The aggressiveness of the blue force has an impact on the 
mean exchange ratio. If higher ratios are to be achieved in this model, the blue force 
must be more aggressive. With increased aggressiveness comes higher variance, 
resulting in higher rewards but also higher risks. In situations were commanders cannot 
tolerate the possibility of high losses, a less aggressive force that relies more heavily on 


the UAV for situational awareness and enemy engagements might be more appropriate. 


B. RECOMMENDATIONS FOR FUTURE RESEARCH 

Because no two combat situations are alike, the possible variations of combat that 
one can attempt to model are infinite. This analysis was meant to provide insights, but 
with those insights comes more questions. Listed below are just a few of the potential 
areas of future research stemming from this scenario: 


e This model uses only one UAV due to its small area of operations. What 
would be the impact of using other intelligence-gathering devices, such as 
unmanned ground vehicles and fixed position sensors, in conjunction with 
the UAV? 


° The UAV in this scenario is invulnerable to enemy fire and is assumed to 
be always operational. While statistics show that the downing of UAVs is 
rare, the slow speeds required in urban environments might make it more 
likely to be shot down. How would this vulnerability affect the results? 


° What would be the effect of limiting the time on station of the UAV due to 
endurance limitations or competing requirements from other units? 
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e This model assumes fairly strict ROE and therefore does not take into 
account the combined arms capability of our forces. How would modeling 
this capability affect the outcome? 


e This model looked at only one performance measure, the exchange ratio. 
Future studies might include other performance measures involving 
civilian casualties or time to complete the mission. 


° Other loss functions may be better suited for combat analysis. Any future 
work using robust design should consider using a loss function that is the 
most appropriate for the MOE being considered. 


With a little imagination the list of variations is virtually endless. This thesis provides 
some preliminary insights into the capabilities the VWTUAV should possess. Other 
questions will arise over the next few years as the VTUAV is developed and fielded. 
Once its capabilities are fully known, the research focus can turn to the best ways to 


employ this new asset. 
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APPENDIX. 
STATISTICS 


Aes MEAN EXCHANGE RATIO MODEL 


Summary of Fit 


RSquare 0.912113 
RSquare Adj 0.891832 
Root Mean Square Error 0.021783 
Mean of Response 0.908221 
Observations (or Sum Wests) 33 


Analysis of Variance 


Source DF Sum of Squares Mean Square 
Model 6 0.12804066 0.021340 
Error 26 0.01233737 0.000475 
C. Total 32 0.14037803 


Parameter Estimates 
Term 


Intercept 

UAV Sweep Width 

UAV CommkReliability 

UAV InfoAccuracy 

BlueAggresiveness 

(UAV CommReliability-50.0606)*(UAV InfoAccuracy- 
50.0606) 

(UAV Sweep Width-135.061)*(UAV Sweep Width- 
135.061) 


F Ratio 
44.9725 
Prob > F 
<.0001 


Estimate Std Error 


0.7597702 0.014026 
0.0000713 0.000053 
0.0015703 0.000136 
0.0012232 0.000136 
0.0014883 0.000348 
0.0000218 0.000005 


-0.000004 8.651e-7 


Effect Tests 

Source Nparm DF Sum of Squares F Ratio 
UAV Sweep Width 1 1 0.00085761 1.8073 
UAV CommReliability 1 1 0.06349981 133.8206 
UAV InfoAccuracy 1 1 0.03852984 81.1985 
BlueAggresiveness 1 1 0.00868285 18.2984 
UAV CommkReliability*UAV InfoAccuracy 1 1 0.00921669 19.4234 
UAV Sweep Width*UAV Sweep Width 1 1 0.00933128 19.6649 
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FINAL REGRESSION MODELS SUMMARY 


t Prob>ltl 
Ratio 
54.17 

1.34 
11.57 
9.01 
4.28 
4.41 


<.0001 
0.1904 
<.0001 
<.0001 
0.0002 
0.0002 


-4.43 0.0001 


Prob >F 
0.1904 
<.0001 
<.0001 
0.0002 
0.0002 
0.0001 


B. LN(VARIANCE) EXCHANGE RATIO MODEL 


Summary of Fit 


RSquare 0.848906 
RSquare Adj 0.8066 
Root Mean Square Error 0.136172 
Mean of Response -1.95967 
Observations (or Sum Wsts) 33 


Analysis of Variance 


Source DF Sum of Squares Mean Square 
Model 7 2.6045342 0.372076 
Error 25 0.4635717 0.018543 
C. Total 32 3.0681059 


Parameter Estimates 
Term 


Intercept 

UAV Sweep Width 

UAV CommReliability 

UAV InfoAccuracy 

BlueAggresiveness 

(UAV Sweep Width-135.061)*(UAV InfoAccuracy- 
50.0606) 

(UAV Sweep Width-135.061)*(UAV Sweep Width- 
135.061) 

(UAV CommReliability-50.0606)*(UAV 
CommReliability-50.0606) 


Effect Tests 
Source 
UAV Sweep Width 1 
UAV CommkReliability 

UAV InfoAccuracy 

BlueAggresiveness 

UAV Sweep Width*UAV InfoAccuracy 

UAV Sweep Width*UAV Sweep Width 
UAV CommReliability*UAV CommReliability 





ne ee 
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1 


a ee 


F Ratio 
20.0657 
Prob > F 
<.0001 


Estimate Std Error 


-2.500952 0.09381 
0.0005978 0.000331 
0.0067098 0.000849 
0.0053155 0.000849 
0.0061833 0.002175 
0.0000303 0.000014 


-0.000023 0.000005 


-0.00013 0.000041 


t Prob>ltl 


Ratio 


-26.66 


1.80 
7.91 
6.26 
2.84 
2.13 


-4.25 


-3.16 


<.0001 
0.0834 
<.0001 
<.0001 
0.0088 
0.0433 


0.0003 


0.0041 


Nparm DF Sum of Squares F Ratio Prob > F 


0.0603143 3.2527 
1.1594423 62.5277 
0.7276172 39.2397 
0.1498617 8.0819 
0.0840276 4.5315 
0.3347629 18.0535 
0.1846960 9.9605 


0.0834 
<.0001 
<.0001 
0.0088 
0.0433 
0.0003 
0.0041 
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